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ABSTRACT 

This  paper  concerns  the  use  of  new  media  technologies,  including  virtual  worlds  and  web  2.0,  for  on-line 
collaborative  activities,  and  specifically  for  the  provision  of  expert  advice  about  the  response  to  large-scale  crises. 
Internet  technologies  in  general  offer  rich  possibilities  for  interactions  involving  remote  experts;  however,  the 
diversity,  novelty  and  power  of  these  technologies  are  such  that  to  introduce  them  into  problem-solving  episodes 
without  first  developing  a  model  of  the  nature  of  those  episodes  and  the  type  of  collaborative  support  they  require, 
risks  confusing  and  discouraging  users.  After  a  brief  discussion  of  the  nature  of  distributed  collaboration  and  the 
implications  this  has  for  any  technical  support,  we  describe  a  virtual  collaboration  environment  that  has  been 
developed  to  foster  task-focused  communities  and  support  them  through  specific  problem-solving  episodes,  and 
present  some  of  the  results  of  evaluation  experiments. 
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INTRODUCTION 

Large-scale  crisis  response  situations  require  collaboration  among  individuals  belonging  to  different  organizations 
and  having  different  backgrounds,  training,  procedures  and  objectives.  Communication  networks  offer  the 
possibility  to  call  upon  expertise  from  around  the  planet  to  assist  during  emergencies;  however,  coordinating  the 
activities  of  geographically  dispersed  experts  to  form  a  coherent  response  raises  a  number  of  questions  about  the 
most  effective  use  of  communication  technologies,  especially  where  these  are  novel  and  have  not  been  designed  for 
such  activities.  Without  a  more  considered  use  of  technology,  opportunities  for  leveraging  expertise  and  resources 
across  organizations  can  be  haphazard,  and  the  response  to  the  crisis  can  appear  as  chaotic  as  the  crisis  itself. 

Seeking  more  effective  and  efficient  means  to  facilitate  crisis  response,  in  2009  the  US  Joint  Forces  Command 
(USJFCOM)  and  the  US  Army  Research  Laboratory’s  Human  Research  and  Engineering  Directorate  (ARL  HRED) 
launched  a  project  under  the  direction  of  Jeffrey  Hansberger  to  design  and  evaluate  a  Virtual  Collaboration 
Environment  (VCE)  and  to  demonstrate  its  potential  for  distributed  crisis  response  planning.  Involved  in  this  project 
are  groups  from  the  University  of  Edinburgh,  the  University  of  Virginia,  Carnegie  Mellon  University  and  Perigean 
Technologies  LLC,  each  of  which  had  an  existing  and  complementary  interest  in  collaborative  work. 

The  initial  technical  concept  behind  the  VCE  was  to  investigate  the  potential  of  new  media  technologies,  specifically 
social  networking  and  virtual  worlds,  to  provide  a  virtual  environment  that  fosters  community  spirit  and 
collaborative  effort  in  some  particular  field  (a  field  in  which,  we  assume,  there  exists  a  potential  community  of  users 
who  have  complementary  knowledge  or  skills  that  contribute  to  problem-solving;  note  that  this  environment  is  not 
restricted  to  crisis  response  tasks).  Thus  envisaged,  the  VCE  would  have  several  specific  requirements: 

•  The  creation  and  maintenance  of  a  community  of  on-line  users  with  diverse  backgrounds  (including  those  with 
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little  or  no  prior  experience  of  virtual  or  on-line  communities).  In  the  first  instance,  the  VCE  is  intended  to  support 
a  Whole  of  Society  Crisis  Response  (WoSCR)  community,  a  loosely  affiliated  community  of  subject-matter 
experts  and  crisis  responders  drawn  from  international  government  and  civilian  organizations  for  the  purpose  of 
contributing  their  specialized  knowledge  to  crisis  response  planning  activities.  In  the  course  of  the  project  an 
initial  mailing  list  of  1600  people  already  involved  in  such  activities  was  used  to  establish  the  community,  of 
which,  at  the  time  of  writing,  some  300  are  active  within  the  VCE  facilities  that  have  been  provided.  It  contains 
members  from  a  number  of  countries  (although  initially  with  a  strong  US  bias)  drawn  from  the  worlds  of 
government,  business  and  academia. 

•  The  provision  for  the  users  of  mechanisms  for  the  creation  and  long-term  development  of  an  on-line  and  on-going 
body  of  experience,  knowledge  and  debate  about  the  field  in  question. 

•  The  ability  for  users  to  act  collaboratively  to  formulate  agreed  solutions  during  specific  problem-solving  episodes. 

The  experiments  that  were  conducted  to  evaluate  the  VCE  focus  on  the  last  of  these  requirements,  since  the  success 
of  the  VCE  as  a  community-building  tool  can  only  be  evaluated  in  the  long  run.  For  the  experiments,  the  VCE  was 
used  in  responsive  mode,  with  users  asked  to  collaboratively  create  emergency  response  plans.  The  ultimate  aim 
here  was  to  show  that  the  use  of  the  VCE  leads  experts  “to  better  produce  better  solutions”.  In  other  words,  we  are 
interested  in  two  aspects,  namely  that  both  the  quality  of  the  solution  plan  and  the  quality  of  the  planning  process  be 
improved.  Of  course,  and  especially  in  experiments,  determining  that  one  proposed  solution  is  better  than  another  is 
difficult;  we  can  however  look  to  more  objective  contents  of  the  plans  such  as  the  aspects  that  they  cover.  Similarly, 
when  comparing  the  quality  of  two  processes  we  must  look  to  other  measures  to  try  to  assess  the  efficiency  of  the 
planning  and  the  contributions  of  participants. 


Technologies  for  Collaboration 

The  typology  commonly  used  to  differentiate  technologies  within  the  Computer- Supported  Cooperative  Work 
(CSCW)  community  categorizes  them  according  to  “their  abilities  to  bridge  time  and  to  bridge  space”  (Baecker  et 
al.,  1995).  Collaborations  can  occur  either  with  participants  physically  collocated  or  else  at  remote  locations.  Here 
we  are  exclusively  interested  in  remote  collaborations.  Similarly,  all  participants  may  collaborate  at  the  same  time, 
or  the  participants  may  work  in  their  own  time.  The  former  requires  some  mode  of  “synchronous  communication”, 
the  latter  “asynchronous  communication”.  In  this  case,  collaborations  in  the  VCE  would  have  two,  quite  different, 
aspects:  a  continuous  asynchronous  collaboration  among  users  to  discuss  and  develop  on-line  documentation 
pertaining  to  their  field  of  interest  (activities  which  would  also  help  foster  a  sense  of  community);  and  interspersed 
synchronous  problem-solving  collaborations  of  relatively  short  duration  in  which  their  expertise  is  put  into  practice. 

These  considerations  would  influence  the  precise  nature  of  the  VCE;  more  concretely,  they  were  embodied  in  the 
decision  to  use  some  combination  of  web  2.0  and  virtual  worlds  technologies.  The  term  “web  2.0”  denotes  an 
approach  to  generating  web  content  that  is  always  open  to  the  possibility  of  -  and  explicitly  encourages  -  the 
contribution  and  collaboration  of  others.  As  such,  a  web  2.0  web  page  might  be  distinguished  from  a  conventional 
web  page  by  its  content  being  (more  or  less  explicitly)  the  result  of  the  interplay  of  the  activities  of  multiple  authors 
or  agents,  by  its  openness  and  by  the  explicit  mechanisms  it  provides  for  contributing  content.  Indeed,  one  might  go 
further  and  suggest  that  much  of  the  appeal  of  social  networking  sites,  blogs,  wikis  and  so  on  lies  in  the  means  they 
give  us  to  assert  our  own  presence  as  member  of  some  community  and  to  have  it  later  affirmed  by  the  interaction  (by 
the  commenting,  supplementing,  contradicting,  erasing)  of  others  in  an  on-going  editing  process.  In  other  words, 
web  2.0  encourages  and  rewards  collaboration,  and  as  such  represents  an  exciting  development  for  those  interested 
in  exploiting  the  potential  of  joint  activity. 

However  one  limitation  (from  the  perspective  of  the  VCE)  of  the  web  2.0  world  is  its  general  assumption  that 
contributions  occur  in  a  more  or  less  temporally  linear  sequence:  one  post  follows  and  responds  to  another.  Here, 
however,  we  require  some  mechanism  for  the  simultaneous  presence  (and  hence  simultaneous  collaboration)  of 
individuals.  For  certain  types  of  interactions  and  collaborations  -  those  that  require  timely  and  agreed  decisions,  or 
the  validation  or  authority  conferred  by  the  presence  of  certain  individuals  -  shared  presence  still  remains  a 
necessity.  While  this  sort  of  interaction  might  be  provided  by  teleconferencing  or  internet  messaging/telephony  tools 
such  as  Skype,  in  this  paper  we  present  the  idea  that  virtual  world  spaces  can  better  fulfil  this  role.  Specifically,  we 
argue  that  virtual  spaces  can  enhance  the  same-time/different-place  interaction  of  existing  tele-  and  video- 
conferencing  tools  by  providing  synchronous  virtual  meeting  support  within  a  space  that  acts  as  a  persistent 
repository  of  informational  objects  representing  resources  used  and  generated  by  the  collaboration,  and  moreover 


Proceedings  of  the  8th  International  ISCRAM  Conference  -  Lisbon,  Portugal,  May  2011 


2 


Wickler  et  al 


The  Virtual  Collaboration  Environment 


can  do  so  in  such  a  way  as  to  complement  the  rich  potential  of  web  2.0  technologies  as  means  for  asynchronous 
collaboration.  The  general  term  we  use  for  this  virtual  meeting  space  is  an  I-Room. 

Hence,  it  was  envisaged  that  web  2.0  and  virtual  worlds  technologies  together  would  provide  the  technical  backbone 
for  meeting  these  requirements.  As  a  first  step,  in  order  to  validate  these  initial  assumptions,  a  Cognitive  Work 
Analysis  (CWA)  was  performed;  this  is  described  in  the  following  section.  We  then  go  on  to  describe  the  varied 
technologies  that  comprise  this  environment,  and  discuss  an  empirical  evaluation  of  its  use  during  a  realistic  crisis 
response  task. 


RELATED  WORK 

The  idea  of  building  virtual  teams  and  communities  for  emergency  response  has  been  explored  in  a  number  of 
projects.  Recently,  within  the  ISCRAM  community,  (Yao  et  al ,  2010)  have  built  a  system  that  allows  virtual  teams 
of  geographically  distributed  experts  to  create  and  discuss  emergency  scenarios.  Collabbit  (de  Lanerolle  et  al ,  2010) 
is  a  virtual  dashboard  that  that  facilitates  distributed  asynchronous  information  sharing  in  emergency  situations. 
However,  we  have  not  found  work  that  aims  to  support  a  virtual  collaboration  environment  with  a  specific  protocol 
and  tools  to  enact  it.  Perhaps  more  interesting  is  the  work  of  (Diez  et  al ,  2010)  which  describes  a  number  of  design 
criteria  aimed  at  supporting  virtual  communities  of  practice,  several  of  which  are  implemented  in  the  system 
described  here. 


COGNITIVE  WORK  ANALYSIS  OF  DISTRIBUTED  COLLABORATION 

The  design  of  the  VCE  was  guided  by  a  Cognitive  Work  Analysis  (Vicente,  1999;  Lintem,  2009)  of  distributed 
collaboration,  with  the  goal-directed  phases  of  forming,  storming,  norming  and  performing  (Tuckman,  1965) 
providing  a  framework  for  understanding  and  supporting  specific  instances  of  collaborative  tasks. 

Any  group  of  humans  brought  together  to  perform  some  task  can  be  considered  to  constitute  a  cognitive  system:  it 
has  knowledge  and  understanding,  can  plan,  decide,  learn,  and  in  general  solve  problems.  However,  this  system 
invariably  functions  within  a  technological  context:  people  use  technical  artifacts  to  perform  their  tasks  (even  natural 
language  is  such  an  artifact,  properly  speaking),  and,  moreover,  apply  particular,  perhaps  socially  or  culturally 
mediated,  physical  and  mental  techniques  in  their  use  of  these  artifacts.  A  CWA  attempts  to  make  explicit  the 
constraints  that  hold  on  work  -  rather  than  the  task,  hence  maintaining  the  focus  on  the  human  -  that  is  done  within 
complex  cognitive  systems  (in  short,  those  that  have  a  certain  amount  of  unpredictability  and  so  cannot  be 
proceduralized).  It  is  also  normative,  in  the  sense  that  it  is  intended  to  identify  the  technical  and  organizational 
relationships  that  must  be  in  place  for  the  work  to  be  performed  effectively.  In  this  manner,  a  CWA  typically  focuses 
on  how  work  can  be  done  compared  to  other  types  of  task  analyses  that  focus  on  how  work  should  be  done  in  a 
limited  set  of  situations,  which  can  decrease  the  flexibility  and  adaptability  of  the  socio-technical  system.  While  not 
itself  a  design  method,  the  results  of  a  CWA  can  be  used  to  support  particular  design  decisions,  as  was  the  case  here. 

A  CWA  progresses  through  multiple  phases  that  systematically  analyze  the  constraints  on  work,  agents, 
organizations  and  activities.  Here  we  restrict  ourselves  to  discussing  the  results  of  the  first  -  and  probably  the  most 
difficult  and  important  -  phase:  Work  Domain  Analysis. 

This  first  phase  of  the  CWA  involves  identifying  the  activity-independent  constraints  of  the  work  domain;  following 
(Lintern,  2009),  here  this  has  been  done  by  decomposing  the  domain  according  to  five  levels  of  abstraction  arranged 
hierarchically.  This  analysis  results  in  the  map  shown  in  Figure  1,  with  content  as  follows: 

•  Domain  purpose:  the  overarching  goal  to  be  achieved  -  in  this  case,  distributed  collaboration. 

•  Domain  values  and  priorities:  principles  or  qualities  on  which  work  in  the  domain  is  founded  -  in  this  case,  we 
identify  coordination ,  communication  and  activity  awareness  as  essential  components  of  distributed  collaboration. 

•  Domain  functions:  the  realization  of  the  domain  values  and  priorities  (and  fulfillment  of  the  domain  purpose)  as 
abstract  functions  within  the  domain;  in  this  case,  we  refer  to  the  4  stages  of  Tuckman’ s  goal-directed  group 
behaviour. 

•  Physical  functions:  the  realization  of  the  domain  functions  in  terms  of  techniques  (Pinelle,  Gutwin  and  Greenberg, 
2003). 
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•  Physical  objects:  technological  artifacts  that  provide  some  aspect  of  the  identified  physical  functionality,  with 
particular  reference  to  the  opportunities  offered  by  web  2.0  technologies  as  well  as  existing  technologies.  Since 
we  are  trying  to  anticipate  a  collaborative  environment  that  does  not  yet  exist,  or  only  partially,  this  level  of  the 
analysis  is  unavoidably  biased  by  our  interests  and  awareness  of  current  technologies.  The  central  position  of  the 
virtual  3D  meeting  space  as  a  potential  for  supporting  multiple  physical  functions  lends  a  certain  amount  of 
credence  to  this  as  a  potential  tool  for  distributed  collaboration;  however,  this  can  only  be  borne  out  by  its 
successful  use.  It  can  be  seen  that  the  virtual  space  potentially  offers  all  but  one  of  the  physical  techniques  that  are 
provided  by  the  other  principal  ‘synchronistic’  interaction  technology,  video  conferencing  (the  omission  being  the 
loss  of  physical-gesture  communication).  In  addition,  the  virtual  space  offers  the  opportunity  for  (persistent) 
information  access  and  transfer.  On  this  basis,  for  the  VCE,  a  virtual  space  was  chosen  instead  of  video- 
conferencing. 

Overall,  the  resulting  analysis  provides  a  coherent  way  of  ensuring  there  is  a  technical  solution  for  each  of  the 
identified  domain  functions  and  their  physical  manifestations,  while  avoiding  the  gratuitous  introduction  of 
technologies  that  add  little  or  no  useful  functionality. 


*W(MJ  2.0  tOOlS 


Figure  1.  Cognitive  Work  Analysis  Phase  I  -  Work  Domain  Analysis. 


OPENVCE:  VIRTUAL  COLLABORATION  ENVIRONMENT  PACKAGE 

So  with  the  results  of  the  CWA  going  some  way  to  confirm  the  initial  assumptions,  we  could  begin  to  put  the 
technologies  in  place  for  the  VCE.  In  this  section  we  discuss  three  of  these  technologies,  namely  a  web-based  portal, 
a  virtual  collaboration  space,  and  the  collaboration  protocols  that  were  introduced  to  guide  the  use  of  the  VCE.  In 
their  generic  forms,  these,  along  with  other  contributory  technologies,  have  been  packaged  as  the  Open  VCE  solution 
-  open,  since  in  the  technologies  chosen  there  was  a  strong  bias  towards  those  that  are  open  source  or  open  access, 
ensuring  that  the  system  as  a  whole  would  be  available  to  as  wide  a  range  of  potential  user  communities  as  possible. 


Community  Web  Portal 

The  VCE  includes  a  web-based  portal  that  would  provide  the  platform  for  the  diachronic  aspects  of  collaboration 
and  communication,  and  for  creating  and  sharing  resources,  as  well  as  more  general  group-building  activity  and 
event  awareness  (http://openvce.net  -  see  Figure  2).  After  some  experimentation  and  discussion  (see 
http://openvce.net/forum-altemative-platforms  and  http://openvce.net/more),  the  open-source  Dmpal®-based 
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software  system  was  adopted  as  the  platform  for  this  site.  Drupal  is  a  widely  used  modular  content  management 
system,  with  an  active  development  community  of  its  own.  It  provides  a  user  management  system  and  web  2.0 
functionality  such  as  user  profiles,  individual  blogs  and  forums.  The  site  was  specialized  with  a  range  of  modules  to 
provide,  for  instance,  twitter-like  activity  awareness,  picture  sharing  and  group  management  facilities  to  allow  ad 
hoc  teams  to  be  constructed  from  among  the  membership  as  a  whole  for  specific  purposes  (such  as  working  on  a 
specific  response  problem).  It  also  includes  mechanisms  that  establishing  relationships  to  the  virtual  space,  allowing 
users  to  associate  their  virtual  personae  with  their  web  profiles,  and  links  to  allow  users  to  “teleport”  into  relevant 
locations  the  virtual  world.  This  site  has  been  augmented  by  a  wiki  (powered  by  the  popular  open  source  MediaWiki 
software),  to  provide  facilities  for  co-authoring  text  documents  (a  facility  felt  to  be  lacking  at  the  time  in  Drupal). 
This  wiki  feature  has  itself  been  supplemented  with  experimental  facilities  for  authoring  and  revising  formalized 
standard  operating  procedures  (Wickler  and  Potter,  2010):  extensions  to  the  wiki  tag-set  allow  the  imposition  of 
meta- information  (objectives,  subtasks)  on  textual  descriptions  of  the  procedure;  this  meta- information  facilitates 
subsequent  searching  and  cross-referencing,  and  allows  the  execution  of  the  procedure  to  be  managed  by  external 
task- flow  tools. 
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Figure  2.  openvce.net  web  portal  home  page. 

The  deployment  and  administration  of  this  web  portal  requires  appropriate  hosting  hardware  and  a  certain  amount  of 
expertise  to  manage  the  site  and  its  users.  This  approach  also  allows  for  additional  functionality  to  be  made 
accessible  to  the  community  by  embedding  appropriate  tools  within  site  pages.  These  tools  can  be  generic 
community  tools  or  introduced  for  specific  tasks. 
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Virtual  Space  for  Intelligent  Interaction:  the  1-Room 

In  addition  to  its  social  and  entertainment  uses,  as  argued  above  virtual  worlds  technology  has  the  potential  to  enrich 
more  serious  forms  of  remote  collaboration.  We  have  developed  these  ideas  into  the  concept  of  the  1-Room  (Tate  et 
ah,  2010).  Put  simply,  an  I-Room  is  an  environment  designed  for  intelligent  interaction.  It  can  provide  support  for 
formal  business  meetings,  tutorials,  project  meetings,  discussion  groups  and  ad-hoc  interactions.  The  I-Room  can  be 
used  to  organize  and  present  pre-existing  information  as  well  as  displaying  real-time  information  feeds  from  other 
systems  such  as  sensor  networks  and  web  services.  It  can  also  be  used  to  communicate  with  participants,  facilitate 
interactions,  and  record  and  action  the  decisions  taken  during  the  collaboration. 


Figure  3.  VCE  web  portal  showing  protocol  support  facilities,  alongside  an  I-Room  virtual  space. 

In  practice,  Second  Life®  and  OpenSim  environments  have  been  used  to  realize  I-Rooms  (Figure  3  shows  an 
I-Room  alongside  a  browser  onto  the  web  portal,  typical  of  how  a  user’s  screen  might  be  laid  out  while  using  the 
VCE).  Using  the  I-Room  concept  within  virtual  worlds  gives  a  collaboration  an  intuitive  grounding  in  a  persistent 
space  in  which  representations  of  the  participants  (their  “avatars”)  appear  and  the  artifacts  and  resources  surrounding 
the  collaboration  can  be  granted  a  surrogate  reality.  Although  for  the  uninitiated  the  virtual  space  can  initially  be 
disorienting  and  video  game-like,  in  our  experience  users  quickly  feel  comfortable  in  the  space  once  any  technical 
issues  are  ironed  out  (as  is  the  case  for  other  video-conferencing  systems,  these  issues  are  usually  related  to  audio 
difficulties  or  firewalls).  Through  an  avatar  a  user  can  see  the  avatars  of  other  users  of  the  space,  and  communicate 
with  those  in  earshot  using  spatialized  voice  (communication  is  also  possible  using  general  text  chat  and  instant 
messaging).  This  audio-visual  positioning  in  3D  space  provides  a  compelling  sense  of  shared  presence  with  any 
other  users  currently  in  the  same  space;  however,  unlike  video-conferencing,  this  medium  lacks  reinforcing  cues 
such  as  eye-contact  (along  with  all  other  forms  of  gestural  communication)  to  confirm  a  speaker  has  the  audience’s 
attention  or  even  that  the  members  of  the  audience  are  actually  ‘in  attendance’  at  their  computers.  Another  difficulty, 
at  least  in  the  case  of  the  chosen  virtual  platforms,  is  the  mutability  of  a  user’s  avatar  -  all  aspects  of  its  appearance, 
even  its  gender,  can  be  changed  on  a  whim  -  could  lead  to  doubt  about  just  whose  presence  one  is  sharing;  and 
although  each  avatar  has  its  name  floating  above  its  head,  since  permitted  names  are  tightly  controlled  to  ensure 
uniqueness  this  is  only  of  limited  value  until  a  sure  association  with  a  human  user  is  made.  The  VCE  includes 
certain  technical  mechanisms,  such  as  the  provision  of  virtual  name-tags,  to  assist  in  this  aspect.  These 
characteristics  of  the  virtual  worlds  threaten  to  undermine  social  structures  such  as  authority  and  trust  that  are 
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grounded  in  identity.  (On  the  other  hand,  they  could  come  to  represent  advantages  of  the  virtual  space,  as  visual 
prejudices  must  be  put  to  one  side.) 

In  addition  to  its  use  as  a  distributed  access  meeting  space,  the  I-Room  can  be  used  to  deliver  intelligent  systems  and 
tool  support  for  meetings  and  collaborative  activities.  In  particular,  the  I-Room  is  designed  to  draw  on  I-X 
technology  (Tate,  2000)  which  provides  intelligent  and  intelligible  (to  human  participants)  task  support,  process 
management,  collaborative  tools  and  planning  aids  to  participants.  This  technology  encourages  collaborators  to 
share  information  about  the  processes  or  products  they  are  working  on  through  a  common  conceptual  model  called 
<I-N-C-A>  (Tate,  2003),  which  describes  a  task  in  terms  of  issues  to  be  addressed,  nodes  (activities)  to  be  included, 
constraints  to  be  respected,  and  annotations  on  these  elements,  and  which  is  based  on  classic  representations  from  AI 
planning,  specifically  hierarchical  task  networks  (Ghallab  et  al ,  2004).  This  framework  allows  access  to  automated 
capabilities  or  agents  in  a  coherent  way,  providing  participants  in  I-Room  meetings  with,  for  instance,  access  to 
knowledge-base  content  and  natural  language  generation  technology. 


Virtual  Collaboration  Protocol 

It  is  one  thing  to  provide  an  appropriate  environment  for  interaction;  it  is  quite  another  to  expect  people  to  use  it 
straight  away  in  the  most  effective  manner,  especially  when  one  considers  the  potential  novelty  of  many  of  the 
technologies  involved.  Furthermore,  the  success  of  collaborations  is  often  determined  to  a  great  extent  by  the 
experience  of  those  involved  and  their  collective  ability  to  organize  their  efforts.  Accordingly,  so  as  to  provide  some 
structure  for  collaborations,  it  has  been  necessary  to  consider  the  use  of  “virtual  collaboration  protocols”,  intended  to 
guide  distributed  collaborative  activities  across  the  diverse  tools  and  organizations  typically  involved  in  crisis 
response.  An  initial  protocol  was  developed  (see  http://openvce.net/vce-protocol)  by  Dr.  Rob  Cross  (University  of 
Virginia)  that  is  intended  to  guide  the  behaviour  of  a  team  comprised  of  WoSCR  members  convened  in  order  to 
provide  expert  advice  to  an  external  agency;  it  is  expected  that  the  request  would  be  of  a  complexity  that  demands 
alternating  virtual  meetings  and  periods  of  diachronic  effort  from  the  team  members.  The  protocol  is  influenced  by 
Tuckman’s  forming-storming-norming-performing  collaboration  model  and  by  how  individuals  communicate  and 
collaborate  through  social  networks  (Cross  and  Parker,  2004).  One  particular  challenge  that  the  protocol  has  to 
accommodate  is  coordinating  efforts  among  a  team  without  an  initially  designated  leader,  which  is  the  case  in  many 
large-scale  crisis  efforts  with  multiple  organizations.  Early  stages,  which  might  be  curtailed  or  omitted  in  subsequent 
collaborations,  aim  to  establish  familiarity  with  the  process,  awareness  of  the  characters  and  abilities  of  the 
participants  and  foster  the  sense  of  ‘group  presence’.  Managerial  roles  are  assigned,  distributing  responsibilities  and 
authorities  among  the  collaborators,  before  the  team  progresses  through  a  series  of  different  tasks  that  decompose 
the  problem  and  then  compose  a  solution. 

To  accompany  this  protocol  tools  based  on  the  <I-N-C-A>  framework  have  been  developed  and  made  accessible 
through  the  web  portal  to  help  track  the  status  of  the  collaboration,  manage  roles,  communicate  with  team  members, 
and  enter  and  share  information.  Furthermore,  a  number  of  standard  operating  procedures  have  been  written  using 
the  Wiki  to  further  decompose  the  subtasks  detailed  in  the  protocol  in  terms  of  the  specific  shared  technologies  that 
might  be  used  to  complete  them  (Wickler  and  Potter,  2010). 


Integartion  of  the  Three  Technologies 

In  the  description  above,  the  three  technologies  are  described  as  independent  components.  However,  significant 
effort  was  spent  to  integrate  these  technologies  to  facilitate  a  coherent  user  experience.  The  central  component 
around  which  this  integration  was  achieved  is  the  virtual  collaboration  protocol.  The  Wiki  with  its  extension  for 
encoding  Standard  Operating  Procedures  was  used  to  make  the  protocol  available  in  a  convenient  hyperlinked  form 
to  users  of  the  VCE.  This  representation  was  also  used  to  provide  a  “to-do”  list  in  the  Drupal  portal  that  was  visible 
to  all  authorized  team  members,  showing  current  progress  towards  their  goals  (this  “to-do”  list  can  be  seen  in  the 
browser  window  on  the  left  of  Figure  3).  In  addition  to  this,  further  assistance  was  made  available  through  the  portal 
for  managing  team  membership,  for  the  assignment  of  roles  to  team  members,  and  for  automatically  generating 
templates  for  the  collection  of  the  information  required  from  team  members  during  the  different  phases  of  the 
protocol.  Where  appropriate,  the  contents  of  these  completed  templates  where  then  used  to  generate  collated 
summaries  of  the  information  gathered,  and  to  provide  ‘presentation  slides’  that  could  be  accessed  and  displayed  on 
‘screens’  in  the  I-Room  to  provide  the  basis  for  discussions  during  subsequent  synchronous  meetings  of  the  team. 
Additional  tools  provided  means  of  exploring  databases  of  experts  where  it  was  recognized  that  the  team  lacked 
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certain  vital  expertise  (a  critical  appraisal  of  the  overall  competence  of  the  team  with  respect  to  the  current  task 
being  itself  part  of  the  protocol).  Neither  the  virtual  spaces  nor  the  collaboration  protocol  assume  a  specific  crisis 
ontology  (Benaben  et  al ,  2008). 


EXPERIMENTS 

The  VCE  attempts  to  facilitate  distributed  collaboration  by  integrating  asynchronous  collaboration  through  web  2.0 
technologies  and  synchronous  collaboration  through  I-Rooms  and  virtual  environments.  Two  experiments  were 
conducted  in  2010  to  examine  the  impact  the  VCE  had  on  crisis  planning  and  collaboration  when  compared  to 
traditional  means  of  distributed  collaboration  among  crisis  response  organizations  and  individuals.  Results  and 
conclusions  from  the  second  and  more  comprehensive  of  the  two  experiments  will  be  discussed  further. 

The  VCE  experiment  introduced  a  biological  agent  outbreak  scenario  to  two  teams  comprising  crisis  expert 
volunteers  distributed  across  the  U.S.,  U.K.,  Canada  and  Italy.  The  traditional  group  (control  condition)  used 
technology  and  means  that  would  normally  be  used  for  distributed  collaboration  across  these  types  of  organizations 
(government,  industry,  non-government,  military,  and  academia)  during  a  crisis,  including  email  for  asynchronous 
and  telephone  and  teleconferencing  for  synchronous  collaboration.  The  virtual  group  (experimental  condition)  used 
the  full  capability  of  the  VCE  as  described  in  this  paper.  The  traditional  group  consisted  of  7  participants  and  the 
virtual  group  had  10  participants;  each  group  had  what  was  considered  equal  expertise  in  crisis  response  and 
biological  outbreaks;  and  no  members  of  either  group  had  any  prior  experience  of  working  together.  Each  group  was 
given  the  same  scenario  and  asked  to  generate  a  crisis  response  plan  over  four  days. 

For  the  virtual  group,  this  meant  following  the  virtual  collaboration  protocol,  which  guided  team  interactions 
through  a  series  of  semi-structured  steps  to  identify  key  crisis  areas,  division  of  labor  and  roles  for  planning 
activities,  and  identify  solutions  to  address  the  various  elements  of  the  crisis.  The  protocol  guided  and  supported 
member  activities  for  both  asynchronous  and  synchronous  interactions.  The  members  of  the  traditional  group,  on  the 
other  hand,  were  free  to  organize  their  activities  over  the  four  days  in  whatever  way  they  thought  fit.  As  such,  their 
success  would  depend  on  their  existing  abilities  to  coordinate  and  collaborate. 
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Figure  4.  Goal  and  procedural  uncertainty  results  from  the  virtual  and  traditional  groups  showing  differences  in 
uncertainty  and  the  mode  each  group  was  interacting  within. 

In  the  event,  both  groups  presented  plausible  solution  plans  as  the  outcome  of  their  collaborations.  However,  initial 
semantic  analysis  results  of  the  final  plans  showed  the  virtual  group’s  final  plan  addressed  more  crisis  response  topic 
areas  and  addressed  them  in  greater  depth  compared  to  the  traditional  group’s  final  plan  (Hansberger,  2010).  In 
order  to  better  understand  these  differences  in  the  final  plan,  we  examined  some  of  the  components  of  collaboration. 
Among  one  of  the  measurements  taken  each  experiment  day  was  a  measure  of  uncertainty  for  each  participant. 
Uncertainty  was  evaluated  along  two  dimensions,  namely  goal  and  procedural  uncertainty.  Goal  uncertainty  is 
defined  as  the  level  of  ambiguity  a  person  has  about  the  goals  or  objectives  in  their  current  situation  or  task. 
Procedural  uncertainty,  on  the  other  hand,  is  how  much  ambiguity  is  associated  with  the  steps  or  procedures 
necessary  to  accomplish  the  defined  goals.  Two  seven-point  Likert  scale  items  measured  each  uncertainty 
dimension,  which  were  averaged  together.  Choo  (2005)  has  defined  these  uncertainty  dimensions  in  terms  of  their 
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interactions  with  each  other.  The  amount  of  goal  and  procedural  uncertainty  possessed  by  an  individual  and  group 
will  dictate  the  mode  (see  Figure  4,  left)  of  interactions  and  ultimately  the  success  of  the  group. 

Placing  the  results  for  goal  and  procedural  uncertainty  along  the  uncertainty  dimensions  presents  a  clear  picture  of 
how  much  uncertainty  was  involved  for  each  group  (Figure  4,  right).  The  traditional  group  finds  themselves 
interacting  in  the  “anarchy  mode”  where  there  is  ambiguity  with  both  goals  and  procedures.  Group  and  individual 
feedback  after  the  experiment  confirms  this  finding.  There  was  considerable  effort  needed  by  this  group  to  establish 
a  common  ground  and  understanding  within  the  group  before  they  could  engage  in  any  planning  efforts.  This  is  also 
indicative  of  collaboration  efforts  among  many  different  organizations,  involving  people  with  different  backgrounds 
and  expertise,  particularly  when  they  have  not  worked  together  before.  The  virtual  group  using  the  VCE  and 
collaboration  protocol  fared  much  better  and  found  themselves  working  within  the  “relational  mode”  where  goals 
and  procedures  are  clear  and  understood.  The  overall  difference  between  the  two  groups  was  statistically  examined 
using  repeated  measures  analysis  of  variance  (ANOVA)  and  there  was  a  significant  difference  between  the  two 
groups  as  suggested  in  Figure  4  (F  =  10.31,/?  <  .01).  The  virtual  group  had  less  goal  and  procedural  uncertainty  as 
they  collaborated  with  their  colleagues,  which  can  result  in  increased  efficiency  and  performance.  These  findings 
provide  some  evidence  of  the  positive  influence  that  can  be  gained  using  integrated  technologies  to  support  both 
asynchronous  and  synchronous  collaboration  over  space  and  time. 


CONCLUSION 

The  Virtual  Collaboration  Environment  has  been  developed  as  a  means  to  support  the  activities  of  distributed 
communities,  specifically  -  but  not  exclusively  -  the  activities  of  the  WoSCR  community.  It  is  intended  to 
supplement  existing  web  2.0  with  virtual  spaces  that  provide  a  means  for  the  simultaneous  presence  that  is  a 
prerequisite  for  certain  aspects  of  collaborative  activity.  Protocols  and  other  tools  are  provided  to  help  with  the  use 
of  this  environment. 

The  efforts  described  in  this  paper  have  been  driven  by  the  idea  that  these  new  technologies  offer  new  possibilities 
for  task-directed  distributed  collaboration,  and  the  desire  to  experiment  with  these.  The  cognitive  work  analysis  and 
the  results  of  the  experiments  provide  a  certain  amount  of  justification  for  their  use,  but  further  work  is  needed  to  put 
the  technologies  and  the  ways  in  which  they  are  used  on  a  firm  theoretical  footing.  This  need  is  both  confirmed  by 
and  made  more  difficult  by  the  rate  at  which  web  2.0  and  virtual  technologies  are  developing,  and  are  subject  to  the 
vagaries  of  fashion  and  popularity.  Attempts  to  keep  abreast  of  these  developments  have  required  a  great  deal  of 
technical  experimentation  and  back-tracking,  as  we  have  watched  new  technologies  emerge  and  supersede  others, 
some  of  great  potential.  This  has  resulted  in  a  certain  ad  hoc  flavour  to  some  of  our  work.  However,  underpinning  all 
these  efforts  is  the  awareness  that  these  technologies  offer  the  emergent  and  often  incidental  property  of  allowing 
their  users  to  assert  their  presence  as  participants  in  distributed  communities;  and  also  of  the  vital  role  that  presence 
plays  in  any  serious  collaborative  activity. 
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